Driver assistance system with sensor offset correction

ABSTRACT

A driver assistance system of a vehicle includes a vision sensor and a non-imaging sensor sensing forward of the equipped vehicle. The control, responsive to processing of captured image data and to processing of captured sensor data, provides a driver assistance function. The control, via to processing of captured image data and captured sensor data, detects the presence of vehicles. The control may disable at least part of the driving assistance function at least in part responsive to the control detecting a vehicle via one of (i) processing of image data captured by said vision sensor or (ii) processing of sensor data captured by the non-vision sensor, and failing to detect that detected vehicle via the other of (i) processing of image data captured by said vision sensor or (ii) processing of sensor data captured by the non-vision sensor.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 15/291,319, filed Oct. 12, 2016, now U.S. Pat. No. 10,137,904,which claims the filing benefits of U.S. provisional application Ser.No. 62/241,299, filed Oct. 14, 2015, which is hereby incorporated hereinby reference in its entirety.

FIELD OF THE INVENTION

The present invention relates generally to a vehicle vision system for avehicle and, more particularly, to a vehicle vision system that utilizesone or more cameras at a vehicle.

BACKGROUND OF THE INVENTION

Use of imaging sensors in vehicle imaging systems is common and known.Examples of such known systems are described in U.S. Pat. Nos.5,949,331; 5,670,935 and/or 5,550,677, which are hereby incorporatedherein by reference in their entireties.

SUMMARY OF THE INVENTION

The present invention provides a driver assistance system or visionsystem or imaging system for a vehicle that utilizes one or more cameramodules or cameras (preferably one or more CMOS cameras) to captureimage data representative of images exterior of the vehicle, including acamera module configured for attachment at a windshield of a vehicleequipped with the vision system. The system uses a forward viewingcamera and a forward sensing non-imaging sensor and determines if thenon-imaging sensor is functioning properly. If the system determinesthat the non-imaging sensor is not detecting an object associated with acamera-sensed vehicle (ahead of the equipped vehicle), then the systemmay be disabled following a period of time and if the vehicle istraveling above a threshold speed. If the system determines that thenon-imaging sensor is detecting an object associated with acamera-sensed vehicle, but the angular offset between the two sensors isabove a threshold level, the system may correct the offset or maydisable the driver assistance system function (depending on how greatthe offset is).

These and other objects, advantages, purposes and features of thepresent invention will become apparent upon review of the followingspecification in conjunction with the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a plan view of a vehicle with a vision system thatincorporates cameras in accordance with the present invention; and

FIG. 2 is a flow chart of the process of the system of the presentinvention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

A vehicle vision system and/or driver assist system and/or objectdetection system and/or alert system operates to capture images exteriorof the vehicle and may process the captured image data to display imagesand to detect objects at or near the vehicle and in the predicted pathof the vehicle, such as to assist a driver of the vehicle in maneuveringthe vehicle in a rearward direction. The vision system includes an imageprocessor or image processing system that is operable to receive imagedata from one or more cameras and provide an output to a display devicefor displaying images representative of the captured image data.Optionally, the vision system may provide a top down or bird's eye orsurround view display and may provide a displayed image that isrepresentative of the subject vehicle, and optionally with the displayedimage being customized to at least partially correspond to the actualsubject vehicle.

Referring now to the drawings and the illustrative embodiments depictedtherein, a vehicle 10 includes an imaging system or vision system thatincludes a forward viewing camera module 12 that is disposed at andviews through the windshield 14 of the vehicle and captures image dataof the scene exterior and forward of the vehicle (FIG. 1). The cameramodule includes a lens for focusing images at or onto an imaging arrayor imaging plane or imager of the camera. The forward viewing cameraviews through the windshield and forward of the vehicle, such as for amachine vision system (such as for traffic sign recognition, headlampcontrol, pedestrian detection, collision avoidance, lane markerdetection and/or the like). The vision system includes a control orelectronic control unit (ECU) or processor (where the ECU may includethe control and processor) that is operable to process image datacaptured by the camera or cameras and may detect objects or the likeand/or provide displayed images at a display device for viewing by thedriver of the vehicle. The data transfer or signal communication fromthe camera to the ECU may comprise any suitable data or communicationlink, such as a vehicle network bus or the like of the equipped vehicle.

The camera system or camera module of the present invention may utilizeaspects of the systems and/or modules described in InternationalPublication Nos. WO 2013/123161 and/or WO 2013/019795, and/or U.S. Pat.Nos. 8,256,821; 7,480,149; 7,289,037; 7,004,593; 6,824,281; 6,690,268;6,445,287; 6,428,172; 6,420,975; 6,326,613; 6,278,377; 6,243,003;6,250,148; 6,172,613 and/or 6,087,953, and/or U.S. Publication Nos.US-2015-0327398; US-2014-0226012 and/or US-2009-0295181, which are allhereby incorporated herein by reference in their entireties. Optionally,the vision system may include a plurality of exterior facing imagingsensors or cameras, such as a rearward facing imaging sensor or camera,a forwardly facing camera at the front of the vehicle, andsidewardly/rearwardly facing cameras at respective sides of the vehicle,which capture image data representative of the scene exterior of thevehicle.

The system also includes at least one other forward sensing sensor 16,such as a radar sensor or laser sensor or lidar sensor (such as a 3Dlidar sensor capturing a three dimensional point cloud of sensing data)or ultrasonic sensor or the like. The other forward sensor capturessensing data that is processed by a data processor of the system todetect or determine the presence of objects ahead of the vehicle.

Many advanced driver assistance systems (ADAS) require informationregarding the location and motion of subject and preceding vehicles andthe environment of the roadway these vehicles are traveling.Determination of object location and motion may utilize various sensingtechnologies including, for example, imaging, light, electromagnetics,ultrasound, and/or the like. In many applications, two sensors areutilized to reduce the risk of false detections by means of redundancy.These sensors may be mounted inside and/or outside of the vehicle.Typically, a forward viewing camera is utilized in ADAS applicationswhere forward looking sensing is required. Such a camera is typicallylocated behind the windshield and views through the windshield of thevehicle. Radars and/or lasers are typically located in areas forward ofthe vehicle radiator.

Combining captured data from two sensors that are mounted atsignificantly different locations can be challenging. The sensor dataprocessing needs to comprehend sensor location differences based on avehicle reference point and occurrence of sensor misalignment. Sensormisalignment determination and correction has a great potential ofoccurrence for the sensor mounted in an area that can be subject toimpacts, such as, for example, in front of the vehicle radiator. Suchmisalignment may result in incorrect association of object informationderived from sensors mounted in different vehicle locations.

To avoid these situations, features that utilize sensors located indifferent vehicle locations require sensor offset determination andcorrection. Also, because these sensors may be very sensitive to angularalignment, this offset can cause errors in the measurement of thelateral position of surrounding objects and thus can lead to unintendedbehavior of the ADAS (which functions responsive to a determinedlocation and movement of an object detected in a field of view of acamera and field of sensing of a non-imaging sensor), associated withnot taking action when it is appropriate or a false action such asactive emergency braking when it is not warranted. If the magnitude ofthe correction exceeds the performance capability of the sensor theassociated ADAS application is disabled.

The present invention utilizes an object association algorithm whichassigns objects from one sensor (such as the camera 12) to objects fromanother sensor (such as the non-imaging sensor 16) and integrates sensoroffset compensation functionality. The compensation functionality checksif a sensor offset is present and corrects the sensor offset ifnecessary. Sensor misalignment is calculated and corrected bycontinuously calculating the mean average m of the angle measurementdifference between both sensors (θ₁, θ₂) for the closest in-path vehicle(CIPV) according to Equation 1 below.

$\begin{matrix}{{m_{k + 1} = {{\frac{n - 1}{n}m_{k}} + {\frac{1}{n}\left( {\theta_{1} - \theta_{2}} \right)}}};} & \left( {{Equation}\mspace{14mu} 1} \right)\end{matrix}$where n represents the number of measurements that is incorporated inthe mean value, m_(k+1) is the mean value of the current time step, andm_(k) represents the mean value of the previous time step.

Assuming that the average difference of angle measurements between bothsensors for a sufficiently long time period is zero, in case nomisalignment is present, sensor misalignment can be detected bycomparing the average error to a threshold constant K₁. This informationmay be used to disable certain functions or to compensate the sensoroffset by adding the calculated offset to the sensor measurement.

FIG. 2 shows a block diagram of the sensor offset correction algorithmof the present invention. The mean angle error m and number ofmeasurements n are initialized with 0. In each time step, the algorithmchecks if a valid camera sensed CIPV (closest in-path vehicle, such as avehicle traveling on the road ahead of the equipped or subject vehicleand thus in the field of view/sensing of the camera and sensor) isavailable and a valid radar sensed object is associated with thisobject. In this case, the number of measurements n is incremented andthe mean angle error according to Equation 1 (above) is updated based oncamera angle θ₁ and radar angle θ₂. If the number of measurements nexceeds the threshold K_Fusion_Offst_TmStps, a valid sensor offset isavailable. K_Fusion_Offst_TmStps defines the measurement duration thatis taken into account for calculation of mean angle error. If n exceedsthis threshold and the mean angle error m is greater than a firstthreshold level (K_Fusion_Offst_Thr1), the sensor offset is corrected bym. In case the mean angle error is greater than a second threshold level(K_Fusion_Offst_Thr2), the ADAS function is disabled.

After a valid sensor offset has been calculated (i.e.,n>K_Fusion_Offst_TmStps and m<K_Fusion_Offst_Thr1 or Thr2), the meanangle error m and the number of measurements n are reset to 0 and thecalculation is repeated. In case no valid radar object is associatedwith the camera CIPV, a counter is incremented. If this counter exceedsa threshold level (K_disable_Thr) and the vehicle speed is greater thana threshold speed (K_min_speed), the system is disabled. Otherwise, thealgorithm checks again for a valid camera CIPV and a valid association.The Counter is reset when a radar object is associated with camera CIPV.The system is disabled because the radar is not detecting an objectassociated with a camera CIPV for a prolonged period of time, and thus,either the radar may be misaligned beyond a correctable state or isotherwise not functioning properly. Responsive to such determination ofradar malfunctioning, the sensing system may be disabled or the driverassistance function may be disabled or the driver assistance functionmay only utilize processing of image data captured by the camera and maynot rely on processing of sensor data sensed by the radar sensor.Similarly, if the system determines that image processing of image datacaptured by the camera fails to detect the presence of a vehicle orobject detected via processing of sensor data sensed by the radarsensor, the system may disable the camera or image processor or thedriver assistance function may only utilize processing of sensor datasensed by the radar sensor and may not rely on processing of image datacaptured by the camera.

Thus, the present invention is operable to disable the ADAS function orsensing system when a determination is made that there is a sensoroffset that may result in erroneous detections or measurements. Thesystem determines if there is a valid radar object detection associatedwith a camera-detected closest in path vehicle. If the system determinesthat there is no such valid radar object detection, then the sensingsystem (at least the radar part of the sensing system) is disabled aftera threshold number of such determinations and if the vehicle speed isgreater than a threshold value. If the system determines that there issuch a valid radar object detection associated with a camera-sensedCIPV, but the angle error is above a threshold level, then the ADASfunction may be disabled to avoid erroneous actions by the ADASfunction.

The system thus uses a vision-based (camera) system and anon-vision-based (e.g., radar or lidar or the like) system. The imagedata captured by the camera(s) and the sensor data sensed by the radarsensor(s) are provided to a control, and the image data and sensor datamay be fused at the control. The control, responsive to processing ofimage data captured by the camera, is operable to detect the presence ofa vehicle exterior of the equipped vehicle and in the forward field ofview of the camera and to determine a vision-based angle of thedetermined vehicle relative to the equipped vehicle. Also, the control,responsive to processing of sensor data sensed by the non-imagingsensor, is operable to detect the presence of the vehicle (the vehicledetermined to be present via image processing of captured image data)exterior of the equipped vehicle and to determine a sensor-based angleof the determined vehicle relative to the equipped vehicle. The anglesmay represent a determined angle between a current directional headingof the vehicle and a center of the detected vehicle or any other pointon the detected vehicle that can be determined by both image processingof captured image data and processing of sensed sensor data.

The control determines an angle difference between the vision-basedangle (the angle of the determined vehicle relative to the equippedvehicle as determined via processing of image data captured by thecamera) and the sensor-based angle (the angle of the determined vehiclerelative to the equipped vehicle as determined via processing of sensordata sensed by the non-imaging sensor). The control disables a driverassistance system function at least in part responsive to the determinedangle difference being greater than a disable threshold level (such asgreater than, for example, a two degree difference or greater than, forexample, a five degree difference or any suitable or selected thresholdoffset). The control may also disable the non-imaging sensor system atleast in part responsive to the control not determining, when processingof sensor data sensed by the non-imaging sensor, the presence of thedetermined vehicle for a period of time.

In other words, if both sensing systems detect a vehicle, but indicatethat the vehicle is at a different location (offset angle) relative tothe equipped vehicle (so as to be at a different angle relative to thevehicle), with the different angle being greater than a disablethreshold angle, then the control disables the driver assistance systemfunction, since this is indicative of one of the systems beingmisaligned such that the captured image data and sensed sensor datacannot be trusted. Also, if both sensing systems detect the vehicle, butindicate that the vehicle is at a different angle relative to thevehicle, with the different angle being less than the disable thresholdangle, but greater than a correction threshold angle (such as greaterthan about 1 degree but less than about 5 degrees or any other suitableor selected range of offset), then the system provides an offsetcorrection (which comprise an average of the vision-based angle and thesensor-based angle over a period of time that is used for determiningthe location of the detected vehicle or object relative to the equippedvehicle, or optionally which may be used to adjust or calibrate thecamera and/or the non-imaging sensor to accommodate or correct for thedetermined offset) and continues to process captured image data andsensed sensor data.

For example, if a driver assistance system function (such as automaticemergency braking or adaptive cruise control or lane departure warningor the like) is responsive to a detected vehicle and location and motionof that vehicle relative to the equipped vehicle, the system of thepresent invention may disable that function when the angle error ordifference (between the vision-based angle and the sensor-based angle)is greater than a threshold amount, but may use an average of thevision-based angle and the sensor-based angle for determining thelocation of the detected vehicle for that driver assistance systemfunction when the angle error or difference is less than the disablethreshold amount. Moreover, if the non-imaging sensor fails to evendetect the presence of the vehicle detected by the imaging system, thenthe system disables the non-imaging sensing system, since this isindicative of the non-imaging sensor not functioning properly, such thatthe sensed sensor data cannot be trusted.

Thus, in accordance with the present invention, image data captured bythe forward viewing camera that views forward through the vehiclewindshield is provided to the control, wherein, via image processing ofcaptured image data by an image processor of the control, the controlcontrols a plurality of driver assistance functions, including, forexample, at least three selected from (i) automatic headlamp control,(ii) lane keeping assist, (iii) traffic sign recognition, (iv) forwardcollision warning, (v) automatic emergency braking, and (vi) pedestriandetection. For driver assistance functions such as forward collisionwarning, automatic emergency braking and pedestrian detection, thecontrol preferably also receives and processes sensor data sensed by anon-imaging sensor (such as a radar sensor or a lidar sensor) of theequipped vehicle. For driver assistance functions such as automaticheadlamp control, lane monitoring/lane keeping, and traffic signrecognition, these driver assistance functions are provided by imageprocessing of captured image data without need to process non-image datasensed by any non-imaging sensor of the equipped vehicle.

In accordance with the present invention, when the control determinesthat the angle difference between the vision-based angle and thesensor-based angle is greater than the disable threshold level, thecontrol disables one or more driver assistance functions (such asforward collision warning, automatic emergency braking and pedestriandetection) and preferably alerts the driver that such driver assistancefunctions have been disabled, so that the driver relies on his/her owndriving abilities for safe maneuvering of the equipped vehicle. When thelikes of an automatic emergency braking driver assistance function isdisabled because of unreliability in angle measurement/estimation fromthe equipped vehicle to a detected other vehicle, other driverassistance functions that do not rely on or require such angledetermination (such as automatic headlamp control or lane departurewarning or traffic sign recognition) need not be disabled. Thus, whenthe data being processed may not be reliable, the system disables driverassistance functions such as forward collision warning, automaticemergency braking and pedestrian detection that rely on sensing orsensor fusion of image data and sensor data, and may not disable otherdriver assistance functions that do not rely on image data and sensordata.

The system may utilize sensors, such as radar or lidar sensors or thelike. The sensing system may utilize aspects of the systems described inU.S. Pat. Nos. 8,027,029; 8,013,780; 6,825,455; 7,053,357; 7,408,627;7,405,812; 7,379,163; 7,379,100; 7,375,803; 7,352,454; 7,340,077;7,321,111; 7,310,431; 7,283,213; 7,212,663; 7,203,356; 7,176,438;7,157,685; 6,919,549; 6,906,793; 6,876,775; 6,710,770; 6,690,354;6,678,039; 6,674,895 and/or 6,587,186, and/or International PublicationNo. WO 2011/090484 and/or U.S. Publication No. US-2010-0245066 and/orU.S. provisional applications, Ser. No. 62/330,557, filed May 2, 2016,Ser. No. 62/313,279, filed Mar. 25, 2016, Ser. No. 62/303,546, filedMar. 4, 2016, and/or Ser. No. 62/289,441, filed Feb. 1, 2016, which arehereby incorporated herein by reference in their entireties.

The camera or sensor may comprise any suitable camera or sensor.Optionally, the camera may comprise a “smart camera” that includes theimaging sensor array and associated circuitry and image processingcircuitry and electrical connectors and the like as part of a cameramodule, such as by utilizing aspects of the vision systems described inInternational Publication Nos. WO 2013/081984 and/or WO 2013/081985,which are hereby incorporated herein by reference in their entireties.

The system includes an image processor operable to process image datacaptured by the camera or cameras, such as for detecting objects orother vehicles or pedestrians or the like in the field of view of one ormore of the cameras. For example, the image processor may comprise animage processing chip selected from the EYEQ™ family of image processingchips available from Mobileye Vision Technologies Ltd. of Jerusalem,Israel, and may include object detection software (such as the typesdescribed in U.S. Pat. Nos. 7,855,755; 7,720,580 and/or 7,038,577, whichare hereby incorporated herein by reference in their entireties), andmay analyze image data to detect vehicles and/or other objects.Responsive to such image processing, and when an object or other vehicleis detected, the system may generate an alert to the driver of thevehicle and/or may generate an overlay at the displayed image tohighlight or enhance display of the detected object or vehicle, in orderto enhance the driver's awareness of the detected object or vehicle orhazardous condition during a driving maneuver of the equipped vehicle.

The vehicle may include any type of sensor or sensors, such as imagingsensors or radar sensors or lidar sensors or ladar sensors or ultrasonicsensors or the like. The imaging sensor or camera may capture image datafor image processing and may comprise any suitable camera or sensingdevice, such as, for example, a two dimensional array of a plurality ofphotosensor elements arranged in at least 640 columns and 480 rows (atleast a 640×480 imaging array, such as a megapixel imaging array or thelike), with a respective lens focusing images onto respective portionsof the array. The photosensor array may comprise a plurality ofphotosensor elements arranged in a photosensor array having rows andcolumns. Preferably, the imaging array has at least 300,000 photosensorelements or pixels, more preferably at least 500,000 photosensorelements or pixels and more preferably at least 1 million photosensorelements or pixels. The imaging array may capture color image data, suchas via spectral filtering at the array, such as via an RGB (red, greenand blue) filter or via a red/red complement filter or such as via anRCC (red, clear, clear) filter or the like. The logic and controlcircuit of the imaging sensor may function in any known manner, and theimage processing and algorithmic processing may comprise any suitablemeans for processing the images and/or image data.

For example, the vision system and/or processing and/or camera and/orcircuitry may utilize aspects described in U.S. Pat. Nos. 8,694,224;7,005,974; 5,760,962; 5,877,897; 5,796,094; 5,949,331; 6,222,447;6,302,545; 6,396,397; 6,498,620; 6,523,964; 6,611,202; 6,201,642;6,690,268; 6,717,610; 6,757,109; 6,802,617; 6,806,452; 6,822,563;6,891,563; 6,946,978; 7,859,565; 5,550,677; 5,670,935; 6,636,258;7,145,519; 7,161,616; 7,230,640; 7,248,283; 7,295,229; 7,301,466;7,592,928; 7,881,496; 7,720,580; 7,038,577; 6,882,287; 5,929,786 and/or5,786,772, and/or International Publication Nos. WO 2011/028686; WO2010/099416; WO 2012/061567; WO 2012/068331; WO 2012/075250; WO2012/103193; WO 2012/0116043; WO 2012/0145313; WO 2012/0145501; WO2012/145818; WO 2012/145822; WO 2012/158167; WO 2012/075250; WO2012/0116043; WO 2012/0145501; WO 2012/154919; WO 2013/019707; WO2013/016409; WO 2013/019795; WO 2013/067083; WO 2013/070539; WO2013/043661; WO 2013/048994; WO 2013/063014, WO 2013/081984; WO2013/081985; WO 2013/074604; WO 2013/086249; WO 2013/103548; WO2013/109869; WO 2013/123161; WO 2013/126715; WO 2013/043661; WO2013/158592 and/or WO 2014/204794, which are all hereby incorporatedherein by reference in their entireties. The system may communicate withother communication systems via any suitable means, such as by utilizingaspects of the systems described in International Publication Nos.WO/2010/144900; WO 2013/043661 and/or WO 2013/081985, and/or U.S.Publication No. US-2012-0062743, which are hereby incorporated herein byreference in their entireties.

The imaging device and control and image processor and any associatedillumination source, if applicable, may comprise any suitablecomponents, and may utilize aspects of the cameras (such as variousimaging sensors or imaging array sensors or cameras or the like, such asa CMOS imaging array sensor, a CCD sensor or other sensors or the like)and vision systems described in U.S. Pat. Nos. 5,760,962; 5,715,093;6,922,292; 6,757,109; 6,717,610; 6,590,719; 6,201,642; 5,796,094;6,559,435; 6,831,261; 6,822,563; 6,946,978; 7,720,580; 8,542,451;7,965,336; 7,480,149; 5,550,677; 5,877,897; 6,498,620; 5,670,935;5,796,094; 6,396,397; 6,806,452; 6,690,268; 7,005,974; 7,937,667;7,123,168; 7,004,606; 6,946,978; 7,038,577; 6,353,392; 6,320,176;6,313,454 and/or 6,824,281, and/or International Publication Nos. WO2009/036176; WO 2009/046268; WO 2010/099416; WO 2011/028686 and/or WO2013/016409, and/or U.S. Pat. Publication Nos. US 2010-0020170 and/orUS-2009-0244361, which are all hereby incorporated herein by referencein their entireties.

The camera module and circuit chip or board and imaging sensor may beimplemented and operated in connection with various vehicularvision-based systems, and/or may be operable utilizing the principles ofsuch other vehicular systems, such as a vehicle headlamp control system,such as the type disclosed in U.S. Pat. Nos. 5,796,094; 6,097,023;6,320,176; 6,559,435; 6,831,261; 7,004,606; 7,339,149 and/or 7,526,103,which are all hereby incorporated herein by reference in theirentireties, a rain sensor, such as the types disclosed in commonlyassigned U.S. Pat. Nos. 6,353,392; 6,313,454; 6,320,176 and/or7,480,149, which are hereby incorporated herein by reference in theirentireties, a vehicle vision system, such as a forwardly, sidewardly orrearwardly directed vehicle vision system utilizing principles disclosedin U.S. Pat. Nos. 5,550,677; 5,670,935; 5,760,962; 5,877,897; 5,949,331;6,222,447; 6,302,545; 6,396,397; 6,498,620; 6,523,964; 6,611,202;6,201,642; 6,690,268; 6,717,610; 6,757,109; 6,802,617; 6,806,452;6,822,563; 6,891,563; 6,946,978 and/or 7,859,565, which are all herebyincorporated herein by reference in their entireties, a trailer hitchingaid or tow check system, such as the type disclosed in U.S. Pat. No.7,005,974, which is hereby incorporated herein by reference in itsentirety, a reverse or sideward imaging system, such as for a lanechange assistance system or lane departure warning system or for a blindspot or object detection system, such as imaging or detection systems ofthe types disclosed in U.S. Pat. Nos. 7,881,496; 7,720,580; 7,038,577;5,929,786 and/or 5,786,772, which are hereby incorporated herein byreference in their entireties, a video device for internal cabinsurveillance and/or video telephone function, such as disclosed in U.S.Pat. Nos. 5,760,962; 5,877,897; 6,690,268 and/or 7,370,983, and/or U.S.Publication No. US-2006-0050018, which are hereby incorporated herein byreference in their entireties, a traffic sign recognition system, asystem for determining a distance to a leading or trailing vehicle orobject, such as a system utilizing the principles disclosed in U.S. Pat.Nos. 6,396,397 and/or 7,123,168, which are hereby incorporated herein byreference in their entireties, and/or the like.

Optionally, the vision system may include a display for displayingimages captured by one or more of the imaging sensors for viewing by thedriver of the vehicle while the driver is normally operating thevehicle. Optionally, the vision system (utilizing the forward facingcamera and a rearward facing camera and other cameras disposed at thevehicle with exterior fields of view) may be part of or may provide adisplay of a top-down view or birds-eye view system of the vehicle or asurround view at the vehicle, such as by utilizing aspects of the visionsystems described in International Publication Nos. WO 2010/099416; WO2011/028686; WO 2012/075250; WO 2013/019795; WO 2012/075250; WO2012/145822; WO 2013/081985; WO 2013/086249 and/or WO 2013/109869,and/or U.S. Publication No. US-2012-0162427, which are herebyincorporated herein by reference in their entireties.

Changes and modifications in the specifically described embodiments canbe carried out without departing from the principles of the invention,which is intended to be limited only by the scope of the appendedclaims, as interpreted according to the principles of patent lawincluding the doctrine of equivalents.

The invention claimed is:
 1. A driving assistance system for a vehicle,said driving assistance system comprising: a vision sensor disposed at avehicle equipped with said driving assistance system so as to have afield of view at least forward of the equipped vehicle, said visionsensor capturing image data; a non-vision sensor disposed at theequipped vehicle so as to have a field of sensing at least forward ofthe equipped vehicle, said non-vision sensor capturing sensor data;wherein captured image data and sensed sensor data are provided to acontrol; wherein said control comprises at least one data processor;wherein image data captured by said vision sensor and sensor datacaptured by said non-vision sensor are processed at said control;wherein said control, responsive to processing at said control of imagedata captured by said vision sensor and responsive to processing at saidcontrol of sensor data captured by said non-vision sensor, provides adriving assistance function for the equipped vehicle; wherein saidcontrol detects presence of vehicles exterior of the equipped vehicleand in the field of view of said vision sensor via processing at saidcontrol of image data captured by said vision sensor; wherein saidcontrol detects presence of vehicles exterior of the equipped vehicleand in the field of sensing of said non-vision sensor via processing atsaid control of sensor data captured by said non-vision sensor; andwherein said control disables at least part of the driving assistancefunction at least in part responsive to said control detecting a vehiclevia processing of image data captured by said vision sensor and failingto detect that detected vehicle via processing of sensor data capturedby said non-vision sensor.
 2. The driving assistance system of claim 1,wherein the at least part of the driving assistance function that isdisabled by said control comprises the entire driving assistancefunction.
 3. The driving assistance system of claim 1, wherein the atleast part of the driving assistance function that is disabled by saidcontrol comprises processing of captured sensor data for vehicledetection so that said control provides the driving assistance functionresponsive to processing of image data captured by said vision sensor.4. The driving assistance system of claim 1, wherein, when a vehicle isdetected by processing of image data captured by said vision sensor andwhen said control fails to detect that detected vehicle via processingof sensor data sensed by said non-vision sensor, said control disablesprocessing of sensed sensor data and provides the driving assistancefunction responsive to processing of image data captured by said visionsensor.
 5. The driving assistance system of claim 4, wherein, responsiveat least in part to a vehicle speed of the equipped vehicle beinggreater than a threshold speed, said control disables processing ofsensed sensor data and provides the driving assistance functionresponsive to processing of image data captured by said vision sensor.6. The driving assistance system of claim 1, wherein, when a vehicle isdetected by processing of image data captured by said vision sensor andwhen said control fails to detect that detected vehicle via processingof sensor data sensed by said non-vision sensor, said control disablesthe driving assistance function.
 7. The driving assistance system ofclaim 6, wherein said control disables the driving assistance functionresponsive at least in part to a vehicle speed of the equipped vehiclebeing greater than a threshold speed.
 8. The driving assistance systemof claim 1, wherein captured image data and sensed sensor data are fusedat said control.
 9. The driving assistance system of claim 1, whereinsaid vision sensor comprises a CMOS camera.
 10. The driving assistancesystem of claim 9, wherein said CMOS camera is disposed at a windshieldof the equipped vehicle so as to have a forward field of view throughthe windshield.
 11. The driving assistance system of claim 1, whereinsaid non-vision sensor comprises a radar sensor.
 12. The drivingassistance system of claim 1, wherein said non-vision sensor comprises alidar sensor.
 13. The driving assistance system of claim 1, wherein saidnon-vision sensor is disposed at a front portion of the equippedvehicle.
 14. A driving assistance system for a vehicle, said drivingassistance system comprising: a vision sensor disposed at a vehicleequipped with said driving assistance system so as to have a field ofview at least forward of the equipped vehicle, said vision sensorcapturing image data; a non-vision sensor disposed at the equippedvehicle so as to have a field of sensing at least forward of theequipped vehicle, said non-vision sensor capturing sensor data; whereincaptured image data and sensed sensor data are provided to a control;wherein said control comprises at least one data processor; whereinimage data captured by said vision sensor and sensor data captured bysaid non-vision sensor are processed at said control; wherein saidcontrol, responsive to processing at said control of image data capturedby said vision sensor and responsive to processing at said control ofsensor data captured by said non-vision sensor, provides a drivingassistance function for the equipped vehicle; wherein said controldetects presence of vehicles exterior of the equipped vehicle and in thefield of view of said vision sensor via processing at said control ofimage data captured by said vision sensor; wherein said control,responsive to processing at said control of sensor data captured by saidnon-vision sensor detects presence of vehicles exterior of the equippedvehicle and in the field of sensing of said non-vision sensor viaprocessing at said control of sensor data captured by said non-visionsensor; wherein, said control disables at least part of the drivingassistance function at least in part responsive to said controldetermining a misalignment of said non-vision sensor relative to saidvision sensor; and wherein said control determines misalignment of saidnon-vision sensor relative to said vision sensor responsive to (i)detecting a vehicle at a first location via processing of image datacaptured by said vision sensor, (ii) detecting that detected vehicle ata second location via processing of sensor data captured by saidnon-vision sensor, and (iii) determining that the second location of thedetected vehicle is different from the first location by a thresholdamount.
 15. The driving assistance system of claim 14, wherein the atleast part of the driving assistance function that is disabled by saidcontrol comprises the entire driving assistance function.
 16. Thedriving assistance system of claim 14, wherein the at least part of thedriving assistance function that is disabled by said control comprisesprocessing of captured sensor data for vehicle detection so that saidcontrol provides the driving assistance function responsive toprocessing of image data captured by said vision sensor.
 17. The drivingassistance system of claim 14, wherein, when the second location of thedetected vehicle is determined to be different from the first locationby the threshold amount, said control disables processing of sensedsensor data and provides the driving assistance function responsive toprocessing of image data captured by said vision sensor.
 18. The drivingassistance system of claim 17, wherein, responsive at least in part to avehicle speed of the equipped vehicle being greater than a thresholdspeed, said control disables processing of sensed sensor data andprovides the driving assistance function responsive to processing ofimage data captured by said vision sensor.
 19. The driving assistancesystem of claim 14, wherein, when the second location of the detectedvehicle is determined to be different from the first location by thethreshold amount, said control disables the driving assistance function.20. The driving assistance system of claim 19, wherein said controldisables the driving assistance function responsive at least in part toa vehicle speed of the equipped vehicle being greater than a thresholdspeed.
 21. The driving assistance system of claim 14, wherein saidcontrol, via processing of image data captured by said vision sensor,determines a vision-based angle of the detected vehicle relative to theequipped vehicle, and wherein said control, via processing of sensordata captured by said non-vision sensor, determines a sensor-based angleof the detected vehicle relative to the equipped vehicle, and whereinsaid control determines an angle difference between the determinedvision-based angle and the determined sensor-based angle, and whereinsaid control disables the driving assistance function at least in partresponsive to the determined angle difference being greater than adisable threshold angle.
 22. The driving assistance system of claim 21,wherein, responsive at least in part to the determined angle differencebeing greater than a correction threshold angle and less than thedisable threshold angle, said control provides a corrected angle of thedetected vehicle relative to the equipped vehicle for use with thedriving assistance function.
 23. The driving assistance system of claim22, wherein said control provides the corrected angle after a thresholdperiod of time during which the determined angle difference is greaterthan the correction threshold angle and less than the disable thresholdangle.
 24. The driving assistance system of claim 14, wherein capturedimage data and sensed sensor data are fused at said control.
 25. Thedriving assistance system of claim 14, wherein said vision sensorcomprises a CMOS camera.
 26. The driving assistance system of claim 25,wherein said CMOS camera is disposed at a windshield of the equippedvehicle so as to have a forward field of view through the windshield.27. The driving assistance system of claim 14, wherein said non-visionsensor comprises a radar sensor.
 28. The driving assistance system ofclaim 14, wherein said non-vision sensor comprises a lidar sensor. 29.The driving assistance system of claim 14, wherein said non-visionsensor is disposed at a front portion of the equipped vehicle.
 30. Adriving assistance system for a vehicle, said driving assistance systemcomprising: a vision sensor disposed at a vehicle equipped with saiddriving assistance system so as to have a field of view at least forwardof the equipped vehicle, said vision sensor capturing image data; anon-vision sensor disposed at the equipped vehicle so as to have a fieldof sensing at least forward of the equipped vehicle, said non-visionsensor capturing sensor data; wherein captured image data and sensedsensor data are provided to a control; wherein said control comprises atleast one data processor; wherein image data captured by said visionsensor and sensor data captured by said non-vision sensor are processedat said control; wherein said control, responsive to processing at saidcontrol of image data captured by said vision sensor and responsive toprocessing at said control of sensor data captured by said non-visionsensor, provides a driving assistance function for the equipped vehicle;wherein said control detects presence of vehicles exterior of theequipped vehicle and in the field of view of said vision sensor viaprocessing at said control of image data captured by said vision sensor;wherein said control detects presence of vehicles exterior of theequipped vehicle and in the field of sensing of said non-vision sensorvia processing at said control of sensor data captured by saidnon-vision sensor; and wherein said control disables at least part ofthe driving assistance function at least in part responsive to saidcontrol detecting a vehicle via processing of sensor data captured bysaid non-vision sensor and failing to detect that detected vehicle viaprocessing of image data captured by said vision sensor.
 31. The drivingassistance system of claim 30, wherein the at least part of the drivingassistance function that is disabled by said control comprises theentire driving assistance function.
 32. The driving assistance system ofclaim 30, wherein the at least part of the driving assistance functionthat is disabled by said control comprises processing of captured imagedata for vehicle detection so that said control provides the drivingassistance function responsive to processing of sensor data captured bysaid non-vision sensor.
 33. The driving assistance system of claim 30,wherein, when a vehicle is detected by processing of sensor datacaptured by said non-vision sensor and when said control fails to detectthat detected vehicle via processing of image data sensed by said visionsensor, said control disables processing of sensed image data andprovides the driving assistance function responsive to processing ofsensor data captured by said non-vision sensor.
 34. The drivingassistance system of claim 33, wherein, responsive at least in part to avehicle speed of the equipped vehicle being greater than a thresholdspeed, said control disables processing of sensed image data andprovides the driving assistance function responsive to processing ofsensor data captured by said non-vision sensor.
 35. The drivingassistance system of claim 30, wherein, when a vehicle is detected byprocessing of sensor data captured by said non-vision sensor and whensaid control fails to detect that detected vehicle via processing ofimage data sensed by said vision sensor, said control disables thedriving assistance function.
 36. The driving assistance system of claim35, wherein said control disables the driving assistance functionresponsive at least in part to a vehicle speed of the equipped vehiclebeing greater than a threshold speed.
 37. The driving assistance systemof claim 30, wherein captured image data and sensed sensor data arefused at said control.
 38. The driving assistance system of claim 30,wherein said vision sensor comprises a CMOS camera.
 39. The drivingassistance system of claim 38, wherein said CMOS camera is disposed at awindshield of the equipped vehicle so as to have a forward field of viewthrough the windshield.
 40. The driving assistance system of claim 30,wherein said non-vision sensor comprises a radar sensor.
 41. The drivingassistance system of claim 30, wherein said non-vision sensor comprisesa lidar sensor.
 42. The driving assistance system of claim 30, whereinsaid non-vision sensor is disposed at a front portion of the equippedvehicle.